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FOREWORD 


Th#  Army  Rnttrch  Inititut*  for  th«  BahavtortI  and  Social  Scianoat  (ARI)  hai  developad  a 
Mrida  ranga  of  lUtittical  modalt  to  test  hypothaiat  ganeratad  in  ralation  to  an  aqually  wide  range 
of  maaiuramant  and  evaluation  eituationa.  Tht  powerful  Randomiaad  Block  (RB)  design  has 
traditionally  bean  a prafarrad  modal  for  much  psychological  research.  The  RB  design  has  the 
stringant  requiramant,  however,  that  the  sample  population  be  strictly  defined  and  stratified 
beforehand,  a requirement  more  appropriate  in  a controlled  laboratory  environment  than  in  many 
Army  field  situations.  This  Technical  Paper  describes  the  development  of  an  alternative  statistical 
design  which  provides  the  edvantagM  of  the  dassic  RB  method  without  its  operational 
disadvantages,  and  which  will  be  useful  not  only  in  the  Individual  Training  artd  Performance 
Evaluation  Technical  Area  in  which  it  was  developad  but  in  other  areas  of  behavioral  science 
research. 

The  entire  research  is  responsive  to  requirements  of  ROTE  Project  2Q762717A745,  Selection 
and  Individual  Training  Research,  FY  197S  Work  Program,  and  to  special  requirements  of  the 
Deputy  Chief  of  Staff  for  Personnel. 


Technical  Director 


A MODEL  FOR  USING  QUALITATIVE  VARIABLES  AS  COVARIATES  IN  THE 
ANALYSIS  OF  COVARIANCE 

BRIEF 


Requirement; 

To  develop,  as  an  alternative  to  the  traditional  Randomized  Block  (RB)  two-way  analysis  of 
variance  design,  an  equally  efficient  statistical  model  that  will  eliminate  the  RB's  requirement  for  a 
priori  stratification  and  sampling  and,  at  the  same  time,  retain  the  RB's  ability  to  handle 
categorical  concomitant  variables.  That  is,  to  develop  a statistical  design  with  the  advantages  of  the 
classic  RB  method  without  its  operational  disadvantages. 


Procedure: 

The  statistical  model  selected  for  comparison  and  test  was  a modified  analysis  of  covariance 
(ANCOVA)  design  that  does  not  require  previously  selected  stratified  samples  artd  does 
incorporate  the  ability  to  handle  categorical  variables-the  Categorical  Analysis  of  Covariance 
'CANCOVA).  The  powers  of  fixed  effects  RB  and  CANCOVA  using  qualitative  (categorical) 
r -ncomitant  variables  were  analytically  and  empirically  compared.  A Monte  Carlo  program 
simulated  fixed  effects  analysis  with  two  levels  of  treatment,  one  criterion  variable,  and  a 
qualitative  concomitant  variable  with  three  design  types.  The  parameters  which  varied  for  each 
design  type  were  sample  size,  ratio  of  numbers  of  row  observations,  eta,  and  magnitude  of 
treatment  effects. 

Findings: 

With  relatively  large  samples,  the  RB  and  the  CANCOVA  designs  yielded  the  same  information 
in  terms  of  component  sums  of  squares.  With  small  samples,  the  power  relationship  is  a function  of 
sample  size,  design  type  and  amount  of  heterogei  eity.  Empirically,  no  practical  difference  was 
found  between  the  ,,uwer  of  RB  and  CANCOVA  when  the  samples  are  large. 


Utilization  of  Findings: 

Where  the  population  cannot  be  well  defined  or  stratified  because  of  necessary  administrative 
and/or  physical  constraints,  practical  field  experimentation  can  be  undertaken  with  a precision 
comparable  to  the  more  expensive  and  laborious  traditional  RB  design.  The  CANCOVA  requires 
only  that  the  subjects  be  sampled  directly  from  the  population  end  randomly  assigned  to  the 
different  experimental  treatments. 
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Experimental  results  are  not  only  affected  by  treatments  but  by 
extraneous  variation  which  often  tends  to  mask  the  primary  effects  of 
experimental  treatments.  The  effect  of  extraneous  variation  on  experi- 
mental results  Is  referred  to  as  experimental  error  variance.  In  the 
behavioral  sciences  experimental  error  variance  can  be  relatively  large 
and  Influence  the  results  of  an  experiment  In  such  a manner  that  only 
large  treatment  effects  can  be  detected,  and  even  these  may  be  subject 
to  uncertainty.  By  the  careful  design  of  experiments,  It  Is  possible 
to  control  sources  of  extraneous  variation,  reducing  experimental  error 
variance  and  Increasing  the  precision  of  the  experiment  (precision 
refers  to  the  power  or  ability  of  a design  to  detect  treatment  effects). 

One  way  of  controlling  experimental  error  variance  Is  by  capitalizing 
on  relationships  between  the  experimental  or  dependent  variables  and 
external  concomitant  variables.  External  concomitant  variables  are 
variables  which  are  measured  prior  to  experimentation  and  are  not 
affected  by  the  experimental  treatment.  For  example,  IQ  could  be  used 
as  an  external  concomitant  varlabl : to  control  for  error  variance  due  to 
difference  In  Innate  abilities.  Tiro  designs  which  employ  external 
concomitant  variables  to  control  error  variance  are;  (1)  Randomized 
Block  (RB),  and  (2)  Analysis  of  Covariance  (ANCOVA) . ^e  RB  experi- 
mentally controls  error  variance  by  using  the  external  concomitant 
variable  to  stratify  the  samples  assigned  to  the  treatment  categories 
Into  homogeneous  groups  called  blocks,  while  the  ANCOVA  statistically 
controls  error  variance  by  using  the  linear  regression  of  the  dependent 
experimental  variable  on  the  external  concomitant  variable. 

Several  researchers  have  compared  the  precision  of  RB  and  ANCOVA 
designs.  Cochran'  found  precision  was  directly  related  to  the  correlation 
of  the  concomitant  and  dependent  variables.  For  correlations  of  less 
than  0.^1  bhe  use  of  RB  or  ANCOVA  to  increase  the  precision  of  the 
experiment  '>«as  Inconsequential,  but  as  the  correlation  Increases  towards 
unity,  sizeable  Increases  in  precision  are  obtained.  For  large  sample 
sizes,  Cochran  concluded  that  for  experimental  designs  In  which  the 
relationship  between  the  experimental  and  concomitant  variables  was  linear, 
the  precisions  of  the  RB  and  ANCOVA  are  about  the  same.  Cox^  found  that 


' Cochran,  W.  G,  Analysis  of  covariance;  Its  nature  and  uses. 
Blometrll.a.  44,  261-281. 

Cox,  0.  R.  The  use  of  a concomitant  variable  In  selecting  an  experi- 
mental design.  Blometrlka.  19^,  44,,  I5O-I58. 
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Ra  provided  greater  precision  when  the  correlation  between  the  variables 
was  less  than  0.6  and  ANCOVA  provided  greater  precision  only  when  the 
correlation  was  greater  than  0.8.  Cox's  conclusion  applies  to  designs 
with  relatively  small  sample  sizes  and  blocking  levels  assigned  on  the 
basis  of  an  underlying  continuum,  i.e.,  a rank  ordering  of  the  categories 
of  the  blocking  variable.  Feldt^  studied  designs  In  which  each  cell  had 
at  least  two  observations.  (Cox's  designs  only  had  one  observation  per 
cell.)  Feldt  concluded  that  for  correlations  less  than  0.4,  RB  resulted 
In  approximately  equal  or  greater  precision  than  ANCOVA;  for  correlations 
greater  than  0.6  ANCOVA  was  "superior." 

The  findings  of  the  above  studies  are  only  applicable  to  designs  In 
which  the  concomitant  variable  Is  continuous,  I.e.,  a variable  which  can 
take  on  any  value  within  a specified  range.  For  example,  weight  Is  a 
continuous  variable;  It  can  take  on  any  value  within  the  possible  range 
of  values  applicable  to  the  object  being  weighed.  The  results  of  these 
studies  do  not  apply  to  designs  In  which  the  concomitant  variable  Is 
qualitative,  I.e.,  a variable  which  Is  categorical,  in  that  It  cate- 
gorizes or  names;  for  example,  different  modes  of  Instruction,  racial 
differences,  differences  In  geographic  origin,  or  social  class  differences 
are  all  qualitative  variables.  When  the  concomitant  variables  are 
qualitative.  It  Is  traditional  to  use  the  RB  technique.  In  the  RB  design 
the  population  Is  stratified  Into  homogenous  groups  based  on  the  cate- 
gories of  the  qualitative  concomitant  variable.  Once  the  population  has 
been  stratified,  random  samples  of  subjects  are  selected  from  each  strata 
and  assigned  to  the  different  experimental  treatments.  In  a laboratory 
setting,  with  a well-defined  population,  a priori  stratification  of  and 
random  selection  of  subjects  from  the  entire  population  Is  easily  ac- 
complished. However,  In  field  experimentation,  where  the  population  Is 
not  well  defined  and  a priori  stratification  of  the  entire  population 
Is  difficult  due  to  administrative  and  physical  restrictions,  employment 
of  RB  designs  can  be  difficult  or  Impossible.  In  many  situations  the 
possible  gain  In  precision  Is  far  outweighed  by  the  necessary  effort 
and  expense  of  employing  the  RB  design. 

The  ANCOVA  offers  a possible  alternative  to  the  RB  design.  ANCOVA 
does  not  require  a priori  stratification  and  sampling;  the  subjects  are 
sampled  directly  from  the  total  population  and  randomly  assigned  to 
the  different  experimental  treatments.  However,  the  traditional  ANCOVA 
model  was  developed  under  the  assumption  that  the  concomitant  variables 
were  random  and  continuous;  as  such,  the  traditional  ANCOVA  Is  not 
applicable  In  situations  where  the  concomitant  variables  are  qualitative 
and  therefore  categorical.  The  purpose  of  this  research  Is  to  develop  a 
categorical  ANCOVA  (CANCOVA),  I.e.,  an  ANCOVA  which  will  allow  the  use 
of  categorical  variables,  and  to  compare  the  precision  of  the  CANCOVA 
with  the  traditional  RB. 


F'lldt,  L.  S.  A comparison  of  the  precision  of  three  experimental 
designs  employing  a concomitant  variable  Psvchometrlka.  19^,  25. 

555-355- 


REVIEW  OF  RELATED  LITERATURE 


Two  of  the  most  widely  used  techniques  for  Increasing  the  precision 
of  randomized  experiments  are:  (1)  stratificatic  i>  blocking  of 

experimental  samples  (RB),  and  (2)  analysis  of  covariance  (ANCOVA).  This 
review  Is  limited  to  studies  In  which  the  precisions  of  RB  and  ANCOVA 
were  compared. 

Cochran*  showed  that,  for  both  RB  and  ANCOVA,  the  gain  In  precision 
over  completely  randomized  designs  was  a function  of  the  size  of  the 
correlation  coefficient  a between  the  criterion  variable  Y and  the 

concomitant  variable  X.  If  o'  Is  the  experimental  error  variance  when 

y 

no  adjustment  Is  employed,  then  the  adjustment  by  covariance  reduces 
this  variance  to; 


where  f^  Is  error  degrees  of  freedom.  The  factor  Involving  f^  is  needed 

to  take  Into  account  errors  In  the  estimated  regression  coefficient  for 
the  bivariate  sample.  The  adjustment  by  blocking  reduces  a"  to: 


(1  - Pty).  (2) 

From  equations  1 and  2 It  Is  clear  that  for  small  values  of  p (p  < 0.5), 

xy  xy 

the  gain  In  precision  afforded  by  using  either  RB  or  ANCOVA  is  likely  to 
be  Inconsequential,  but  as  p increases  towards  unity,  sizeable  Increases 

In  precision  are  obtained.  Cochran  concluded  that  for  experimental 
designs  In  which  the  relationship  between  X and  Y Is  linear,  the  precisions 
of  the  RB  and  ANCOVA  are  about  the  same  (for  large  sample  sizes). 

Cox  ’ used  two  measures  of  Imprecision  as  a basis  for  comparing  the 
relative  efficiency  of  RB  and  ANCOVA.  The  first,  true  Imprecision  was 
based  on  the  population  value  of  the  average  error  variance  for  the 
difference  between  two  treatment  means  (adjusted  for  covariance  when 
appropriate).  The  second,  apparent  Imprecision  was  defined  as  the 


* Cochran,  1957,  °P*  cit. 
® Cox,  1957,  op.  cit. 
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product  of  the  true  Imprecision  and  an  adjustment  factor^  based  on  error 
degrees  of  freedom.  The  adjustment  factor  allows  for  a more  meaningful 
comparison  of  the  relative  efficiency  of  two  techniques  which  use  the 
same  number  of  observations  but  have  different  error  degrees  of  freedom. 
Symbolically j these  two  indices  are: 


I 

t 


/ (1 


)/n] 


I 


a 


I 

t 


where  y is  the  variance  of  the  estimated  difference  between  a pair 

of  treatment  means  averaged  over  all  pairs  of  treatment  means,  f^  is  the 

error  degrees  of  freedom,  n Is  the  number  of  observations  per  treatment 
group,  a is  the  variance  of  Y within  each  treatment  population,  Is 

the  linear  correlation  coefficient  between  X and  Y,  and  [2o®  (1  - p )/n] 

’ y xy' 

Is  the  minimum  variance  of  the  difference  between  treatment  means  averaged 
over  all  pairs  of  treatment  means.  For  any  pair  of  designs  based  on  the 
same  total  sample  size,  comparison  of  the  respective  values  of  will 

indicate  which  of  the  two  designs  is  more  efficient. 


Cox  evaluated  I and  I for  Rfi  and  ANCOVA  using  several  combinations 
t a 

of  total  sample  size,  p , and  numbers  of  levels  of  treatment  (t).  Cox 

xy 

concluded  that  RB  provided  greater  precision  when  p < 0.6,  and  ANCOVA 

xy 

provided  greater  precision  only  when  p >0.8. 


It  should  be  noted  that  designs  used  by  Cox  were  not  fixed  effects. 
Cox  assumed  that  the  blocking  levels  were  selected  randomly  by  ranking 
subjects  on  the  blocking  variable  X,  subdividing  the  ranked  subjects 
into  t groups,  and  assigning  one  subject  per  block  at  random  to  each  of 
the  t levels  of  treatment.  Thus,  the  interaction  of  blocks  and  treat- 
ment provides  an  appropriate  error  term.  On  the  other  hand,  designs 
used  In  behavioral  research  are  typically  fixed-effects  models  with 
more  than  one  observation  per  cell. 


® Fisher,  R.  A.  The  design  of  experiments.  London:  Oliver  and  Boyd, 

1949. 
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Feldc^  extended  Cox's  study  to  designs  In  which  the  main  and  Inter- 
action effects  were  fixed  and  each  cell  had  at  least  two  observations. 

All  designs  considered  by  Feldt  were  completely  randomized  having  t 
levels  of  treatment  with  an  equal  number  of  observations.  The  samples 
were  assumed  to  have  been  drawn  from  t normally  distributed  populations 
with  constant  variance  and  a normally  distributed  concomitant  X,  linearly 
related  to  Y.  The  treatment  population  means  and  variances  of  the  X 
variable  and  the  within  treatment  correlation  coefficients  between  X and 
Y were  assumer'.  to  be  equal  across  all  treatment  levels. 


Feldt  used  the  same  Indices  of  Imprecision  as  Cox;  however,  they 
were  modified  to  account  for  more  than  one  observation  per  cell.  Feldt 
concluded  that  for  < 0.4  RB  resulted  In  approximately  equal  or 

greater  precision  than  ANCOVA;  for 


> 0.6  ANCOVA  was  "superior."  For 


relatively  high  values  of  p^  and  relatively  small  total  sample  size, 

the  difference  In  precision  In  favor  of  ANCOVA  was  appreciable.  This 
difference  was  attributed  to  the  relatively  small  sample  size  not 
permitting  the  experimenter  to  employ  a sufficiently  large  number  of 
blocking  levels  to  exploit  fully  the  value  of  the  concomitant  variable 
In  RB.  Feldt  noted  that  for  p^  < 0.2  and  small  sample  sizes  neither 

ANCOVA  nor  RB  yielded  appreciably  greater  precision  than  a completely 
randomized  design. 


In  all  articles  reviewed,  the  concomitant  vavlable,  X,  was  assumed 
to  be  normally  distributed.  Further,  none  of  the  articles  was  addressed 
to  the  situation  In  which  the  concomitant  variable  was  qualitative. 
Feldt's  and  Cox's  Indices  of  Imprecision  assume  that  the  control  variable 
had  an  underlying  continuous  distribution  and  that  each  category  of  the 
control  variable  had  a definable  variance.  When  X Is  qualitative  the 
wlthln-block  variance  of  X Is  zero;  therefore,  these  Indices  cannot  be 
used  to  compare  the  precision  of  RB  and  ANCOVA  when  the  concomitant 
variable  Is  qualitative. 


METHODOLOGY 

The  methodology  discussion  is  divided  Into  two  sections,  analytical 
and  empirical.  The  analytical  section  Involves  a comparison  of  the 
power  of  RB  and  CANCOVA  In  which  the  number  of  obseivatlons  (n^^)  within 

each  of  the  I categories  of  the  concomitant  variable  Is  constant.  The 
empirical  section  compares  the  power  of  CANCOVA  In  which  n^  Is  a random 

variable  (RCANCOVA)  with  RB  In  which  n^  is  a constant;  sample  size  is 

held  constant  for  both  designs.  This  comparison  simulates  the  practical 
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Feldt,  1958,  op.  clt. 


circumstances  surrounding  the  probable  Implementation  of  these  designs. 

The  Introduction  of  n.  as  a random  variable  complicates  the  mathematical 
models  making  it  only^feasible  to  compare  the  power  of  the  RB  ani  RCANCOVA 
designs  by  Monte  Carlo' methods . 


Analytical 

Development  of  the  analytical  models  is  based  on  the  General  Linear 
Model  (GIM).  Structural  models  for  RB  and  CANCOVA  were  developed  and 
appropriate  parameter  and  design  matrices  were  defined.  Functional 
relationships  were  established  between  the  RB  and  CANCOVA  model  components 
by  comparing  the  parameter  and  design  .natrices  of  the  respective  models. 
Mathematical  functions  were  developed  for  the  RB  and  CANCOVA  treatment 
effect  F ratios.  The  power  of  the  two  models  was  compared  by  examining 
the  relationship  between  the  mathematical  functions  defining  the  respec- 
tive treatment  effect  F probability  distributions. 


Empirical 

The  Generation  of  Random  Normal  Samples.  The  generation  of  random 
normal  samples  for  RB  and  RCANCOVA  was  accomplished  by  using  RANDN,® 

RANDN  was  called  separately  to  generate  sets  of  observations  for  each 
cell  of  a design.  Each  set  of  observations  was  generated  from  a popu- 
lation with  a specified  mean  and  standard  deviation  of  1.  Cell  means 
were  computed  from  row  mean  values  (row  means  are  a function  of  the 
correlation  coefficient,  eta,  for  the  specific  design  being  run)  and 
treatment  effect  differences.  A FORTRAN  program  was  used  to  compute 
the  values  of  row  means  for  each  of  the  possible  design  combinations 
run;  program  documentation  and  a table  of  the  row  means  used  for  each 
value  of  eta  are  presented  In  Appendix  A. 

Goodness-of-Flt  Tests.  The  randomness  and  goodness  of  fit  to 
normality  of  the  samples  generated  by  RANDN  are  dependent  on  the  Initia- 
tion number  used  in  the  generating  process.  Several  starting  numbers 
were  tested  for  the  fit  of  the  numbers  they  generated  to  a hypothetical 
normal  distribution  by  means  of  a chi-square  goodness-of-flt  test. 
Documentation  for  the  chi-square  program  Is  presented  In  Appendix  B. 

In  addition  to  these  tests,  data  were  generated  based  on  5,000  samples, 
using  each  starting  number,  for  the  12  possible  design  combinations  In 
which  eta  and  treatment  effect  were  both  0.  The  goodness  of  fit  of  the 
empirical  frequency  of  rejection  of  the  null  hypothesis  of  no  treatment 
effect  to  the  expected  frequency  of  rejection  under  the  central  F 
distribution  was  determined  for  six  nominal  alpha  levels  .01,  .05,  .10, 
.25,  .50,  .75  using  a 5"degree-of-freedom  chi-square  goolness-of-f It  test. 


® University  of  Maryland  UNT'vAC  II06  EXEC  8 Math-Pack  users'  guide. 
College  Park,  Md.:  Computer  Science  Center,  University  of  Maryland, 

1970. 


Empirical  Power  Comparisons.  Only  designs  In  which  I was  equal  to 
2,  5i  and  4 were  studied.  Several  combinations  ol  parameters  were  used 
with  each  of  these  three  design  types  ("design  type"  refers  to  the 
number  of  categories  In  the  concomitant  variable) . The  parameters  which 
were  varied  for  each  design  type  were:  (1)  sample  size,  (2)  ratio  of 
number  of  row  observations,  (3)  eta,  and  (4)  magnitude  of  treatment 
effect.  Table  1 llste  the  values  assigned  to  each  parameter  for  each 
of  the  three  design  types. 


Table  1 

PARAMETERS  FOR  EACH  OF  THE  THREE  DESIGN  TYPES 


Design  Type 


Parameter 

Two-Category 

Variable 

Design 

Three-Category 

Variable 

Design 

Four-Category 

Variable 

Design 

Sample  Size 

20 

36 

56 

80 

144 

224 

Marginal  Ratlc 

equal  n 

1:1 

1:1:1 

1:1:1:1 

unequal  n 

4:1 

4:1:1 

4:1:1:1 

Eta 

0.0 

0.0 

0.0 

0.5 

0.5 

0.5 

0.9 

0.9 

0.9 

Magnitude  of 

0.0 

0.0 

0.0 

Treatment  Effect 

0.2 

0.2 

0.2 

0.5 

0.5 

0.5 

Two  sets  of  3i000  sample  data  generations  were  run  for  each  of  the 
48  design  combinations  In  which  the  values  for  eta  and  the  treatment 
effect  were  non-zero.  One  set  of  data  generations  was  analyzed  by  the 
RB  subroutine  and  the  other  by  the  RCANCOVA  subroutine  of  the  Monte 
Carlo  program  (Appendix  C) . For  each  run,  empirical  F ratios  for 
treatment.  Interaction  (RB),  and  heterogeneity  of  regression  (RCANCOVA) 
were  computed  along  with  their  respective  probabilities  under  the 
null  hypotheses  of  no  treatment  effect,  no  Interaction  effect,  and 
homogeneity  of  regression.  Each  of  the  computed  probabilities  for  the 
treatment  effect  F ratios  was  compared  to  the  6 nominal  alpha  levels, 
and  rejection  rates  were  tabulated  for  each  alpha  level.  For  each 
sample  generated,  the  computed  probability  for  Interaction  and 
heterogeneity  of  regression  F ratios  was  compared  to  the  .01  and  .0^ 
alpha  levels;  those  samples  with  Interaction  or  heterogeneity  of 
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regression  significant  at  the  ,0l  and  .0^  levels  were  tabulated  as  sub- 
groups. The  empirical  rejection  rate  for  the  treatment  effect  F ratios 
In  each  subgroup  was  tabulated  In  the  same  manner  as  described  for  the 
total  set  of  samples. 

The  empirical  power  of  a given  design  combination  for  a specified 
alpha  level  Is  equal  to  the  proportion  of  times  the  null  hypothesis  of 
no  treatment  effect  Is  rejected.  The  empirical  powers  of  RB  and  RCANCOVA 
were  compared  at  each  of  the  six  nominal  alpha  levels.  The  statistical 
significance  of  the  comparisons  was  determined  by  using  either  of  two 
statistics;  the  first  Is  a z statistic  defined  by  Walker  and  Lev;^ 


s/Npq/NiNa 


where  Nj  Is  the  number  of  cases  In  population  1 for  which  the  observed 
proportion  of  rejection  Is  p^ , Is  the  number  of  cases  In  population 
2 for  which  the  observed  proportion  of  rejections  Is  p, , p • Pi  + Pa , 
q * 1 - p,  and  N * + Nj  ; the  second  Is  a z'  statistic  defined  by 

Haight:'® 


z 


/ 


y - X 


ry 


+ X 


where  y Is  the  frequency  of  rejections  in  population  1 and  x is  the 
frequency  of  rejections  In  population  2.  The  z statistic  Is  approximately 
a random  variable  with  a normal  distribution  and  the  z'  statistic  Is 
approximately  a random  variable  with  t Poisson  distribution.  Since  the 
binomial  teat  becomes  skewed  and  the  normal  approximation  Is  Inaccurate 
for  proportions  close  to  0 or  1,  the  z statistic  was  used  for  comparisons 
In  which  the  population  proportions  were  close  to  and  the  z'  statistic 
was  used  for  comparisons  In  which  the  population  proportions  were  close 
to  1 or  0. 


RESULTS 


Analytical 

The  following  conventions  were  used  In  developing  the  analytical 
arguments : 


* Walker,  H.  M.,  and  Lev,  J.  Statistical  Inference.  New  York:  Holt, 
Rinehart  and  Winston,  1955* 

'®  Haight,  F.  A.  Handbook  of  the  Poisson  distribution.  New  York: 
Wiley,  1967. 
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1.  Superscripts  designate  the  model  or  design  type:  f « full  model, 

r “ reduced  model,  c » CANCOVA,  and  b « RB. 

2,  Subscripts  Index  a variable  within  a specific  design:  1 - 

category  of  control  variable,  J « level  of  the  treatment  dimension,  and 
k = Individual  within  an  IJ  combination. 


5.  Matrix  notation  Is  condensed  by  writing  a column  vector  as  Its 
transpose  row  vector;  for  matrices  Involving  repetition  of  elements, 
the  following  column  vector  notation  Is  used:  An  Is  a column  vector 


of  Oj  A's ; 


e.g. 


1 Is  a 


column  vector  of  9 I's. 


The  RB  model  Is  a two-way  analysis  of  variance  design  In  which  the 
levels  of  the  blocking  variable  correspond  to  the  categories  of  the 
qualitative  control  variable.  In  the  CANCOVA  model  the  categories  of 
the  qualitative  control  variable  are  translated  Into  sets  of  dichotomous 

covarlables."  If  X (m  • 1,  2,  ...  I-l)  Is  the  m*”^  covarlable  score  In 
in  ' 

the  1^^  (1  1,  2,  ...,  l)  category  of  the  qualitative  control  variable, 

then  the  value  of  X for  m - 1 Is  d and  the  value  of  X for  m / 1 Is  g; 
m ’ in 

the  values  of  d ^ g are  arbitrary,  e.g.,  d » 1 and  g « 0,  or  d » -1  and 
g * 1,  etc.  Table  2 schematically  represents  the  covarlable  allocation 
or  blocking  strategy  for  a design  In  which  the  qualitative  control 
variable  has  I categories. 


ANOVA  designs  can  be  represented  as  special  forms  of  the  general 
linear  model  (GLM).'^  The  matrix  form  for  a sample  of  n scores  based 
on  a model  with  p < n parameters  Is 


Y ■ X e + e , 

where  Y Is  an  (nxl)  vector  of  random  observations,  X Is  an  (nxp)  design 
matilx  of  known  quantities,  ^ Is  a (pxll  vector  of  unknown  parameters, 
and  e Is  an  (nxl)  vector  of  unobserved  random  errors,  normally  distrib- 
uted with  E(£)  = 0 and  E(e  e')  ■ la*,  where  I Is  the  (nxn)  Identity  matrix 
and  a*  Is  the  variance.  Y and  X represent  observable  data,  whereas  9 and 
£ are  unknown.  The  least  squares  estimator  of  Q Is  defined  such  that 

e 'e  ” minimum. 


” Suits,  D.  B.  Use  of  dummy  variables  in  regression  equations.  Journa 1 
of  the  American  Statistical  Association.  December  1962,  548-55I. 

” Dayton,  C.  M.  An  Introduction  to  the  general  linear  model.  University 
of  Maryland,  Department  of  Measurement  and  Statistics  Monograph, 

College  Park,  Maryland,  1969> 


Table  2 


SCHEMATIC  OF  RB  (1x2)  OR  CANCOVA  (1x2  WITH  I-l  COVARIABLES) 


Dichotomous  Dummy 
Covariables 

m 

Treatment 

Xg  ... 

^I-l 

Blocks 

1 

2 

10  ... 

0 

^111 

^121 



. 

1 

• 

• 

10  ... 

0 

'^lln^ 

^12n^ 

• 

• 

• 

• 

• 

a 

0 0 ... 

1 

Y 

1-1,11 

’^I-l,  21 

« • 

• « • « « 

• « 

0 0 ... 

« 

• 

1 

I-l 

Vl.lnj.3^ 

• 

a 

a 

^I-l, 2n^_^ 

^i-r  • 

0 0 ... 

0 

^111 

"l21 

• • 

• • a • • 

• 

I 

• 

a 

a 

V- 

• • 

0 0 ... 

0 

^Ilnj 

^12Hj 

Y.i. 

Y.2- 

10 


I 


The  value  of  ^ which  minimizes  is  found  by  the  solution  of 

d (£^e) 


where  t = 1,  2^  . . . , p.  The  solution  results  in  a system  of  normal 
equations ; 

X x'S  « x'y 


If  X is  nonsingular,  then  (X^X)  ^ exists  and  there  is  a unique  solution 

1 = 

The  model  for  a score  in  the  RB  analysis  is 

ijk  “j  *^1  ^ij  Ijk 


where  the  superscript  (fb)  denotes  full  model  for  RB,  is  an 

additive  constant  or  grand  mean,  (j  “1,  2,  J)  is  the  effect 

of  being  in  the  level  of  the  treatment  dimension,  [i  “ 1,  2. 

...,  I)  Is  the  effect  of  being  In  the  blocking  level,  ^he 

interaction  effect  of  oelng  in  the  IJ*'*'  cell  of  the  design,  and  e[^^ 

th 

is  the  error  effect  associated  with  the  k observation  In  the 

Ij  cell  of  the  design.  If  the  parameter  and  design  matrices  are 
defined  using  this  model,  the  design  matrix  will  be  singular  with 
column  rank  JI.  Glnce  the  smaller  order  of  this  design  matrix  Is  (J+1) 
(I+l),  In  order  to  remove  the  singularity  a total  of  (J+I+1)  restrictions 
Is  needed.  These  restrictions  can  be  generated  from  three  classes  of 
linear  restrictions: 

J 


z a 

J-i 


J 


0 


(5) 


I 

r e 

1-1  ’ 


(4) 


I-  J 
Z Z Y 

1=1  j-1 


iJ 


(5) 
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Under  these  restrictions,  appropriate  parameter  and  design  matrices  are:'® 


fi' 


(fb) 


r (fb)  ( 


fb)  (fb)  (fb) 

>Pl  >^2  ' 


(fb)  (fb)  (fb) 

.Pi_i  »Yii  >y-[2 


. Y 


(fb) 

1,1-1 


] 


Si 

Si 

a a a 

Si 

Si 

Si 

Si 

Si 

Si 

Si 

• a a 

Si 

Si 

Si 

Si 

* 

a 

a 

a 

a 

a 

• 

a 

a 

a 

a 

a 

• 

• 

a 

• 

a 

• 

• 

• 

a 

• 

S-1. 

1 S-1,1 

S-1.1 

S-i.i 

a a a 

S-1,1 

S-1.1 

S-1,1 

a a a 

S-1,1 

.Si 

Si 

Si 

Si 

a a a 

Si 

Si 

Si 

a a a 

Si 

Sa 

■^la 

Sa 

Si 

a a a 

Sa 

’Sa 

Sa 

a a a 

Sa 

Sa 

’Sa 

Sa 

Si 

a a a 

Sa 

Sa 

"Sa 

a a a 

Sa 

a 

a 

a 

a 

a 

a 

a 

a 

a 

a 

a 

• 

a 

a 

• 

a 

* 

• 

• 

a 

• 

S-i. 

3 ^1-1,2 

S-1.  a 

\-1.2 

a a a 

S-1,  a 

S-1,  a 

S-1,2 

a a a 

^i-i,a 

Sa 

■S. 

Sa 

Sa 

a a a 

Sa 

Sa 

Sa 

a a a 

Sa 

'®  To  conserve  space  the  design  and  parameter  matrices  Illustrated  through- 
out the  rest  of  this  paper  are  restricted  to  designs  in  which  the 
treatment  dimension  has  two  levels;  this  restriction  does  not  preclude 
generalization  of  the  results  to  designs  with  more  than  two  levels  of 
the  treatment  dimension. 
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The  model  for  the  same  score  under  the  CANCOVA  analysis  using 
separate  wlthln-rell  regression  Is 

.(he) 


j « + r b X + e' 

'jk  ej^ 


where  (he)  Is  a superscript  denoting  a CANCOVA  design  using  separate 


.(he) 


Is  an  additive  constant, 


wlthin-cell  regression,  p. 
effect  of  being  In  the  level  of  the  treatment  dimension  , 


Is  the 

b Is  the 
th 

regression  coefficient  due  to  the  regression  of  the  1 covarlable  within 

tVi  th 

the  j treatment  level  on  the  Y scores  within  the  j treatment  level, 

X Is  the  value  of  the  1^^  covarlable  score  for  the  observation 
th  (he) 

within  the  J treatment  level,  and  e'  ' Is  the  random  error  for  the 

•*  y 

k observation  within  the  J level  of  treatment.  The  design  matrix 
using  this  model  will  be  singular  with  column  ranx  JI.  Since  the 
smaller  order  of  the  design  matrix  Is  JI+1  to  remove  the  singularity  1 
restriction  Is  needed: 


'J 


Under  this  restriction,  appropriate  parameter  and  design  matrices  are: 


r(hc)_ 


e(hc)' 

f (he)  (he) 

’‘*11’ 

’’2I’” 

•’V: 

^11 

^11  ^11 

Si 

• • • 

Si 

Si 

Si  °"n 

Si 

• * • 
• 

Si 

K 

i.  in 

* 

K , 

1-1,1 

^1-1,1  ^1-1,1 

I-l 

» 1 

I-X 

K 

K 2n  , 

... 

s, 

11 

11  11 

11 

11 

-1 

• 

12 

12 

• 

• 

S2 

-^22 

• 

: 

0 

• 

1-1,2 

1-1,2 

1 

-1 

"12 

"12 

1 

22  »■ 


’1-1.2 


K 

K 

K 

0 


'12 

12 

^12 

•••  V, 

'22 

22 

22 

•••  K , 

1-1, 

2 1-1,2 

I-: 

0 

...  0 

'12 

"12 

"12 
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Both  and  X are  nonsingular  matrices  with  column  rank  JI.  Since 

both  are  based  on  full-rank  models,  they  account  for  the  same  amount  of 

the  total  variance  and  SS 


(fb)  ^ '*■ 


An  alternative  to  the  (he)  CANCOVA  is  a model  In  which  the  pooled 
wlthln-cell  regression  coefficient  Is  used  instead  of  separate  wlthln- 
cell  regression  coefficients.  The  model  for  a score  is 

(c)  fc) 

Y.u  = ^ + a)  ^ + Z b,  X,  ..  + e^, 

jk  j 1 IJk  Jk 

where  (c)  denotes  the  CANCOVA  model  in  which  the  pooled  wlthln-cell 

f c ^ r c ) 

regression  coefficient  Is  used,  p,'  Is  an  additive  constant  a.  Is  the 

th  J 

effect  of  being  in  the  j level  of  the  treatment  dimension,  b^^  Is  the 

pooled  wlthln-cell  regression  coefficient  due  to  the  regression  of  the 

1^^  covariable  on  the  Y scores,  X ..  Is  the  value  of  the  1^^  covariable 
th  th 

score  for  the  k Individual  In  the  j level  of  the  treatment,  and  e,, 

th  th 

is  the  random  error  for  the  k observation  within  the  j treatment 
level.  The  design  matrix  for  this  model  will  be  singular  with  column 
rank  J+I+1.  Since  the  smaller  order  of  this  design  matrix  is  J+I,  in 
ov'der  to  remove  the  singularity,  1 restriction  is  needed: 

J 

- a • 0 

j-i  -1 


Under  this  restriction  the  appropriate  parameter  and  design  matrices 
are : 


9<'>'  ■ 


Scheff^,  H.  The  analysis  of  variance.  New  York:  John  Wiley  & Sons, 

1959. 

Grayblll,  F.  A.  An  Introductlor  to  linear  statistical  models.  New 
York:  McGraw-Hill,  I96I.  Pp.  IO6-I45. 
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Si 

\2 

Si 

... 

Si 

Si 

“"21 

Si 

Si 

• 

* 

• 

• 

• 

• 

• 

• 

• 

• 

S-1.1 

S-1.1 

S-1.1 

0 

~"l-l,l 

^1-1,1 

. Si 

Si 

Si 

Si 

Si 

^12 

■^12 

S2 

S2 

S2 

S2 

■S2 

0 

“"22 

S2 

S2 

* 

* 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

^1-1,2 

^1-1,2 

0 

~”l-l,2 

S-1.2 

... 

S-1,2 

\2 

■\2 

S2 

S2 

... 

S2 

If  the  data  are  such  that  for  the  covariable  - b^  for  j “ 1, 
then  the  (c)  and  the  (he)  CANCOVA  models  are  Identical  and 
account  for  the  same  amount  of  total  variance;  however,  If  for  the  l'^ 
covariable  b for  some  value  of  j then  the  column  rank  of 

IJ  “ 

will  be  greater  than  the  column  rank  of  X'  ' and  the  difference  in  the 
amount  of  total  variance  accounted  for  by  the  two  models  will  be  reflected 
In  the  difference  between  their  respective  error  sum  of  squares In 
general,  the  difference  between  the  error  sum  of  squares  for  the  two 
models  Is 

- SS,  (6) 

e e h ' ' 

where  SS^  Is  Che  sum  of  squares  due  to  heterogeneity. 


A similar  approach  can  be  used  with  the  RB  model  to  define  Che  sum 
of  squares  for  Interaction  (SS^  ■').  If  “ 0 for  all  1,  j,  then  the 
appropriate  parameter  and  design  matrices  for  this  reduced  model  are: 


(rb)  ^(rb)  ^(rb)  ^(rb) 


.^1  .6^ 


p(>^b) 

' ’^I-l 


I 

i 


'®  Grayblll,  I96I,  op.  clt. 
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0 


11  ^11  ^11 


11 

"11 

"11 

21 

\l 

0 

""21 

^(rb) 


I-l.l 

S-1.1 

0 

~"l-l, 1 

0 • • • 

■"l-l.l 

1 

~"l-l,l 

'll 

\l 

0 ... 

■"ll 

of 

'l2 

~\2 

^"12 

0 ... 

■"12 

0 

■"12 

22 

~\2 

■"22 

1 

"22 

0 

■"22 

1-1,2 

1-1,2 

\-1.2 

0 

"l-l,2 

1 

M 

c 

■^"12 

9 

to 

\2 

0 

■"12 

..  0 
■"12 

The  difference  In  column  rank  between  the  full  and  reduced  RB  models  Is 
(j-1)  (I-l).  This  difference  in  column  rank  is  due  to  the  elimination 
of  the  Interaction  component  (y^j  “ O)  reduced  model;  therefore, 

the  difference  between  the  reduced  and  full  models'  error  sum  of  squares 
is  due  to  interaction 


(7) 

e e Y ' 

Functional  relationships  can  be  established  between  the  sum  of  squares 
components  for  the  RB  and  CANCOVA  models  by  examining  the  least  squares 
estimators  for  their  respective  parameter  vectors 

|(c). 

|(rb)  _ X^rb)'^ 

Since  Y is  the  same  score  vector  for  both  models  and 
least  squares  estimators  are  equal: 


- 16 


(8) 


1 

•K. 

r; 

O' 

' 1 

7... 

^(rb) 

7.1- 

(C)  ^ 

- 

•>2 

• 

^2 

• 

7,.. 

• 

• 

• 

/l-l 

• 

From  equation  8 It  can  be  shown  that 


= 

(Q) 

a 

0 

n 

a 

co(rb ; 

ssj 

■ ^4"'’ 

(10) 

By  substituting  the  appropriate  equalities  from  equations  9 and  lo 
Into  equations  6 and  7 it  can  shown  that 


SS  - 
h Y 


and  since  X 


(c)  . jj(rb) 


e e 


(11) 


(12) 


Table  3 llstr  all  the  component  sums  of  squares  for  each  model  along  with 
their  respective  degrees  of  freedom. 


The  difference  In  the  power  of  the  RB  and  CANCOVA  analyses  to  detect 
treatment  effects  Is  dependent  on  both  the  amount  of  Interaction  (hetero- 
geneity) present  In  the  data  and  the  difference  In  degrees  of  freedom 
associated  with  the  mean  square  error  term  (MS^)  under  each  model.  For 

the  (fb)  RB  with  I levels  of  blocking  and  J levels  of  treatment: 


e 


(15) 
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Table  5 


SUM  OF  SQUARES  AND  DEGREES  OF  FREEDOM  FOR 
RB  AND  CANCOVA  ANALYSES 


RB 

CANCOVA 

Sum  of 

Degrees  of 

Sum  of 

Degrees  of 

Squares 

Freedom 

Squares 

Freedom 

SS 

J-1 

SS 

J-1 

a 

a 

I-l 

SS, 

b 

(regression) 

I-l 

SS 

Y 

(J-1)  (I-l) 

SS 

e 

n. . -J-I+1 

SS 

n. .-JI 

ss<" 

c) 

n, .-JI 

e 

e 

(J-1)  (I-l) 

Note.  The  SS^  for  CANCOVA  can  be  partitioned  into  which  is  the  error  sum  of  squares  jsirtg  separate  within-cell 

regression  coefficients,  and  SS^  which  is  the  sum  of  squares  for  heterogenity  of  regression. 


and  for  the  (c)  CANCOVA  analysis  with  I-l  covariables  and  J levels  of 
treatment 


MS 


(c) 


n. .-J-I+1 

The  f ratio  for  the  treatment  effect  in  the  (fb)  RB  is 


(14) 


,(fb) 


a 


MS 


MS 

-fiL- 


(n..-Jl)MS 


SS 


(fb) 


a 


(15) 


e e 

and  the  F ratio  for  treatment  effect  in  the  (c)  CANCOVA  analysis  is 


, . MS 

y(0  = 

« MS^ 
e 


(n..-J-I+l)MSj^ 


SS 


(16) 


- Ifi  - 


For  sufficiently  large  sample  sizes ^ 


'■(4' 


where  P(F  ) is  the  probability  that  F s F under  the  central  F distri- 

' a ^ a a 

bution  when  the  sum  of  squares  for  interaction  (heterogeneity)  is  equal 
to  0.  When  the  sum  of  squares  for  interaction  (heterogeneity)  is  not 
equal  to  0,  the  following  relationships  hold: 


when  F.  = F > 1. 
h y ’ 


(1?) 


where  Fj^  is  the  F ratio  for  testing  the  null  hypothesis  of  homogeneity 
of  regression  and  F^_  is  the  F ratio  for  testing  the  null  hypothesis  of 
no  interaction. 


Y 

:(fb) 

a 


> p 


when  F, 


F < 1. 

a 


Given  that  F^^^  > F^*^^  the  proof  of  equations  I7  and  18  is 


Of 


Q 


(n. .-JI)MS 


SS 


(fb) 


a 


(n..-J-I+l)MS 


SS 


(IS) 


Since  SS 


- SS^^*^^  + SS  then 
e e h’ 


(n..-JI)MS 


SS 


(n..-J-I+l)MS 


SS 


(he)  ^ 


& 


ss^ 


(19) 


Dividing  by  MS^,  expanding  and  combining  like  terms: 


^ lJ-1)  (I-lj 

sg(hc)  (n..-JI) 


(20) 


dividing  both  sides  by  (J-1)  (l-l)  / (n,.-Jl): 


(n.,-Jl)SS 


(J-1)(I-1)SS 


(he) 


> 1 


(21) 


since 


(n,.-JI)SSj^  MSj^ 

(J-l)(l-l)Ss(’’‘^^  Ms('’'^^ 
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I 


p/p(^)\  < p when  = F >1. 

r ) 1“ ) 

From  equations  I7  and  18  it  is  clear  that  when  Fj^  = F-y  “ 1 the  power 
of  the  RB  (Pw(RB))  is  equal  to  the  power  of  the  CANCOVA  (Pw(C)),  when  Fj^  = 
F^  > 1,  Pw(RB)  > Pw(C),  and  when  F^^  = F_^  < 1,  Pw(C)  > Pw(RB). 


The  above  relationships  only  apply  when  the  sample  size  is  sufficiently 
large  to  negate  any  degrees  of  freedom  differences  in  the  distribution 
of  the  treatment  effect  F ratios  for  the  RB  and  CANCOVA  analyses.  When 
the  sample  size  is  relatively  small,  the  degrees  of  freedom  difference 
between  the  two  techniques,  along  with  the  level  of  heterogeneity 
(interaction)  in  the  samples,  plays  an  important  part  in  determining  the 
relative  power  of  the  two  techniques. 


The  following  arguments  are  used  to  define  the  functional  relation- 
ship between  the  amount  of  heterogeneity  (interaction)  in  small  samples 
and  the  relative  power  of  the  RB  and  CANCOVA  techniques.  If  the  critical 

values  at  a given  alpha  level  are  known  for  F^^^^  and  F^*”^  and  = 

(fb)  s a O'  ’ e 

SS^  is  defined  as  a constant  equal  to  1,  then  the  value  of  the 


SS.  = SS 
h Y 


can  be  computed  as  a proportion  of 


SS 


(fb). 


the 


value  of  this  proportion  when  Pw(RB) 


Pw(C)  is  defined  as  a pivot 

point  value  (PV)  for  the  power  function.  When  the  ratio  SS, /SS^^^^ 
Y e 

PV,  then  Pw(RB)  < Pw(C), 


/ f b V n e 

SS''"'"'  / SS'  ' > PV , Pw(RB)  > Pw(C),  but  when  this  ratio  is  less  than 


Table  4 lists  the  PV  values  for  alpha  levels  of  .01  and  .O'  for  two-, 
three-,  and  four-category  control  variables  with  sample  sizes  varying 
from  (J-I*2)  to  (J-I-6)  observations  per  cell.  The  algebraic  argument 
used  to  compute  the  listed  PV  values  is: 


Given  that  SS^^^ 
e 


then 


0; 


(22) 


and 


MS 


1 + SS,_/n.. -.1-1+1 
h 


(23) 
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Table  4 


PIVOT  POINT  VALUE  FOR  THE  .01  AND  .05  ALPHA  LEVELS 


Design  Type 


Total  Sample  Size 


Alpha 


.01  PV .05  PV 


8 

.5899 

.4585 

Two-Category 

12 

.1994 

.1690 

Variable  Design 

16 

.1159 

.1019 

20 

.0791 

.0728 

24 

.0604 

.0566 

12 

.6277 

.5015 

Three-Category 

18 

.2285 

.2040 

Variable  Design 

24 

.1571 

.1270 

30 

.0977 

.0925 

36 

.0770 

.0729 

16 

.6350 

.5093 

Four-Category 

24 

.2377 

.2183 

Variable  Design 

52 

.1382 

.1301 

40 

.0997 

.0982 

48 

.0791 

.0761 

where  and  are  the  critical  values,  at  a given  alpha  level. 

a a , o r , 

of  the  treatment  effect  F ratios  for  the  RB  and  CANCOVA  designs  re- 
spectively, and  n..  Is  the  total  sample  size,  MS  can  be  computed  as  a 

( fb } ^ 

function  of  the  known  constants  F^  , n..,J,  and  I using  the  relation- 
ship defined  In  Equation  22: 

% - (24) 


Substitution  of  K for  MS^  In  Equation  25  results  in 

(c)  K 

a ° 1+SS. /n..-J-I+l  ’ 

h 

transposing  gives 


SS. 


K(n..-J-I+1) 




(25) 


21  - 


I 


Since  PV  = SS. and  •-  = 1,  then  PV  = SS^ ; therefore 

he  e e ' h ’ 

Equation  S')  can  be  used  to  compute  PV  values  for  different  designs. 

Table  4 only  lists  PV  values  for  designs  with  sample  sizes  up  to  a maximum 

of  (J"I-6),  Total  sample  sizes  greater  than  (J«I-6)  are  sufficiently 

large  that  the  differences  in  the  distributions  of  the  treatment  effect  F 

ratios  for  the  RB  and  CANCOVA  techniques  are  negligible  and  the  PV  values 

can  be  computed  directly  from  Equation  2o. 


Empirical 

None  of  the  12  null  design  combinations  (eta  = 0 and  treatment  effect  ° 
O)  produced  significant  lack  of  fit  using  a chi-square  goodness-of-f It 
test.  Table  ^ contains  suinnary  data  for  the  power  comparisons  between 
RB  and  CANCOVA  for  all  48  design  combinations  in  which  eta  and  the 
treatment  effect  were  non-zero.  The  empirical  power  for  each  analysis 
Is  expressed  as  the  proportion  of  samples  In  which  the  null  hypothesis 
of  no  treatment  effect  was  rejected.  The  empirical  powers  for  both  the 
RB  and  RCANCOVA  analyses  are  given  for  the  .01,  .0'),  and  .10  nominal 
alpha  levels  for  each  of  the  48  design  combinations.  The  power  com- 
parisons are  divided  into  three  groupings:  (1)  comparisons  for  the 

total  number  of  samples  generated  (5,000),  (2)  comparisons  for  those 
samples  In  which  interaction  and  heterogeneity  were  significant  at  the 
.01  level,  and  (5)  comparisons  for  those  samples  in  which  interaction 
and  heterogeneity  were  significant  at  the  .0''  level. 


DISCUSSION 


Analytical  Results 

If  Lnere  Is  no  Interaction  or  heterogeneity  of  regression,  the  least- 
square  estimators  of  parameters  and  error  terms  for  RB  and  CANCOVA  are 
Identical.  In  situations  where  there  is  interaction  or  heterogeneity 
of  regression,  the  error  sum  of  squares  for  CANCOVA  can  be  partitioned 
into  a sum  of  squares  for  heterogeneity  which  equals  the  sum  of  squares 
for  interaction  under  RB  analysis,  and  an  error  sum  of  squares  due  to 
the  use  of  separate  withln-cell  regression  coefficients  which  equals 
the  error  sum  of  squares  under  RB  analysis.  Given  the  situation  where 
sample  size  is  fixed  and  the  number  of  observations  per  level  of  the 
control  variable  is  a constant,  RB  and  CANCOVA  provide  the  same  infor- 
mation. 

Even  though  the  two  techniques  can  provide  the  same  information  in 
terms  of  component  sums  of  squares,  the  relative  powers  of  the  two 
techniques  are  not  necessarily  equal.  The  power  relationship  was  shown 
to  be  a function  of  sample  size,  design  type,  and  amount  of  heterogeneity 
(interaction).  For  situations  in  which  the  heterogeneity  (interaction) 

is  equal  tu  zero.  therefore,  P ^ 


- 22  - 


Table  5 

EMPIRICAL  FOUER  LEVELS  EOR  THE  48  DESIGN  COMBINATIONS  IN  WHICH 
ETA  AND  THE  TREADCNT  EFFECT  WERE  NON-ZERO 


TWo-Ceteeorv  Deelgne 
Soplc  Size  ^ 


Marginal  Ratio 

coual 

li 

ineoual 

Treatneot 

Effect 

.2 

• 5 

.2 

.5 

Eta 

■3 

.9 

.3 

•9 

.3 

.9 

.3 

.9 

Model  Power  Coatperleons 
RB/RC 

RB 

.0187 

.0205 

.0973 

.0617 

.0113 

.0147 

.0615 

.0550 

RC 

.0193 

.0117 

.0560 

.0660 

.0153 

.0197 

.0613 

.0613 

Nonlnal 

.0707 

.0683 

.1813 

.1317 

•0520. 

.0753 

.1850 

.1697 

Alpha 

.0723 

.0637 

.1695 

.1750 

. .0675 

.0693 

.1703 

.1795 

rc 

.1233 

.1^ 

.2900 

.2610 

.1100, 

.1303 

•2923, 

.2787 

.1397' 

.1213 

.2790 

.2773 

.1260 

.1315 

.2^ 

.2860 

RBOl/ROPl 

eOOCX) 

.0000 

.1429 

.1795 

.0000 

.0370 

.1250 

.0690 

.0000 

.0000 

.0936 

.0000 

.04  55 

.0315 

.09^ 

.0714 

Nominal 

.1111 

.0508 

.4266 

.3077 

.0345 

.1111 

.4688, 

.2069 

Alpha 

.0833 

.0294 

.1875 

.1071 

.1616 

.0938 

.1935 

.2697 

.2222 

.1765 

.4266 

.4103 

.1379 

.1481 

.7188, 

.2069 

.1667 

.0588 

.3438 

.2145 

.2727 

.1563 

.4194 

.2697 

RBO5/RCO5 

.01  “ 
RC 

.0126 

.0301 

.0725 

.1143 

.0147 

.0325 

.0621 

.1269 

.0064 

.0126 

.0361 

.0897 

.0164 

.0145 

.0405 

.0970 

Nominal 

•05 

^ RC 

.1282 

.0977 

.1664 

.2597 

. 1809 

.0976 

•2795„ 

.2687 

Alpha 

.0641 

.0440 

.1325 

.1697 

.0656 

.0870 

.1216 

.2388 

10  “ 

.1731 

.1729 

.2734 

.3597 

.1838 

.1870 

.4037,, 

.5060 

RC 

.1474 

.U52 

.2590 

.2514 

.1311 

.1522 

■ 1959 

.2761 

SiBple  Size  80 


RB/RC 


.01 

RB 

•0380, 

.0473 

.3207 

.3413 

.0397 

.0505 

.5307 

.3307 

RC 

.0507 

.0480 

.3407 

.3590 

.0420 

.0445 

.5453 

.3410 

Nominal 

•05 

RB 

.1443 

.5603„ 

.6050 

• 1397 

.1510 

.9787 

.5917 

Alpha 

RC 

.1600 

.1417 

.6083 

.5983 

.1347 

.1340 

.5960 

.3897 

.10 

RB 

.2103„ 

.2537 

•6no„ 

.7120 

.2180 

.2347 

.6813 

.7110 

RC 

.2510 

.2210 

.7220 

.7080 

.2247 

.2215 

.6920 

.7073 

RBOl/RCOl 

.01 

RB 

.0789 

.0000 

.2903 

.5000 

.0870 

.0800 

•7300,, 

.9769, 

RC 

.1111 

.0345 

.4091 

.4333 

.0278 

.1000 

.2069 

• 1739 

Nominal 

.05 

RB 

.1979 

.0714 

.5606, 

• 6905 

.1759 

.2000 

1.0000_ 

.7769 

Alpha 

RC 

.2222 

.1034 

.7727 

.6667 

.1944 

.4000 

.5172 

.4348 

.10 

RB 

.2368 

.1429 

.6774 

.6905 

.2174 

.5200 

1.0000,H, 

.7692 

RC 

.2222 

.2414 

.6182 

.7667 

.2778 

.4500 

.7241 

.6087 

RBO5/RCO5 

.01 

RB 

.0426 

.0274 

.2933 

.3893 

.0307 

.0816 

.3462 

.3643 

RC 

.0643 

.04  32 

.51^ 

.5431 

.0405 

.0496 

.4118 

.3660 

Nominal 

.05 

RB 

.1360 

.1436 

.5533 

.6242 

.1304 

.2109 

.3897 

.6214 

Alpha 

RC 

.2071 

.1481 

.6174 

.3693 

.1486 

.1631 

• 3948 

• 9732 

.10 

RB 

.2097 

.2808 

.6200 

.6913 

• 1997 

.2925 

.3897. 

.7214 

RC 

.2714 

.2407 

.7114 

.6788 

.2027 

• 2199 

.7190 

.6667 

23  - 


I 


Table  ^ - ConClnued 


Marginal  Ratio 

Three-Cateeorv  Oealcnt 
Seattle  Sice  36 
equal 

unequal 

Treataent  Effect 

.2  ,5 

.2 

■ 5 

Eta 



-3 J 

J 3 3— 

3 

Hodel  Power  Co^erlsona 
RB/RC 


.01 

RB 

.0165 

.0177 

.1443 

.1210 

.0207 

.0240 

.1197 

.1480 

RC 

.0200 

.0147 

.1307 

.U60 

.0170 

.0190 

.1203 

.1310 

Nonlnal 

.05 

RB 

.0765 

.0747 

.3453 

.2923 

.0827 

.0823 

.3077 

.3110 

Alpha 

RC 

.0863 

.0763 

.3277 

.2980 

.0790 

.0887 

.2953 

.3035 

.10 

RB 

.1520 

• 15?7 

.484  3„ 

.4183 

.1523 

.1510 

.4223 

.4377 

RC 

.1537 

.1407 

.4500 

.4117 

.1493 

.1487 

.4155 

.4233 

RBOl/ROOl 

.01 

RB 

.0667 

.U76 

.3000 

0833 

.115* 

• 1379, 

.0690 

• 5000„ 

RC 

.0000 

.0000 

.1500 

• 0370 

.0345 

.0263 

.1333 

.1143 

Noalnal 

.05 

RB 

.2000, 

.1765 

.5500 

.2085 

.1923 

• 1379 

.3448 

•5000.. 

Alpha 

RC 

.0645 

.0952 

.3500 

.1852 

.1724 

.1316 

.3333 

.1714 

.10 

RB 

.3000 

.4118 

.6500 

.2500 

.2692 

.1379 

.3793 

.5000 

RC 

.0968 

.1429 

.4000 

.2593 

.2759 

.1842 

.4333 

.3429 

RB05/R005 

.01 

RB 

.0301 

.0567 

.2444. 

.1293 

.0556 

.0460 

.2418. 

.2690. 

RC 

.0136 

.0204 

.1437 

.1020 

.0252 

.0291 

.1497 

.1523 

Nocalnal 

■05 

RB 

.1566, 

.1277 

.4444 

.5265 

.1528 

.1034 

.6l44„ 

.3655 

Alpha 

RC 

.0816 

.0884 

.3436 

.2517 

.0881 

.1047 

.3293 

.3113 

.10 

RB 

.2349 

.2482, 

.5704 

.59‘»6 

.2431 

.1667 

■7320.. 

.5172 

RC 

.1565 

.1361 

.4500 

.3197 

.1?72 

.2035 

.>•551 

.4437 

Senple  Site  Ikk 


RB/RC 


.01 

RB 

.0847 

.0860 

.6400 

• '4373 

.0847 

.0793 

.6550 

.62H 

RC 

.0830 

.0817 

.6600 

.(427 

.0873 

.0853 

.6460 

.6397 

Nominal 

.05 

RB 

.2240 

2210 

.8433 

.8347 

.2230 

.2120 

.8363 

.8430 

Alpha 

RC 

.2247 

.2.’37 

.8400 

.8313 

.2250 

.2180 

.8977 

.8423 

.10 

RB 

.3227 

.3173 

.9040 

.9897 

.3245 

.3193 

.9140 

.9123 

RC 

.3323 

.3267 

.9077 

.9080 

.3337 

.3273 

.9177 

.9073 

RBOl/ROOl 

.01 

RB 

.0000 

.1667 

.^55 

.6800 

.1379 

.0541 

.5152 

.5000 

RC 

.0000 

.0870 

.6522 

.7333 

.1818 

.0750 

.3415 

.6585 

Nominal 

.05 

RB 

.0556, 

.4333 

.9091 

.7200 

.2414 

.1692 

.7976 

.7368 

Alpha 

RC 

.3^ 

.2174 

.7609 

.8667 

.3182 

.2300 

.7073 

.8973 

.10 

RB 

.1111 

.5000 

• 9545 

.8400 

.2759 

.2973 

.9697 

.9737 

RC 

.3600 

.3043 

.8478 

.9333 

.3636 

.3500 

■ 8337 

.9756 

RB05/RC05 

.01 

RB 

.0682 

.0701 

.6395 

.7105 

.1500, 

.0588 

.6017 

.6493 

RC 

.0511 

.0612 

.6187 

.699* 

.0687 

.0963 

.5633 

.6434 

Nominal 

.05 

RB 

.1894 

.2739 

.8971 

.8750 

.7929 

.1830 

.8559 

.8881 

Alpha 

RC 

.2117 

.2585 

.7937 

.8675 

.2977 

.2444 

.8608 

.8601 

.10 

RB 

.2197, 

.3694 

.9252 

.9211 

.3786 

.2876 

.9237 

.9776 

RC 

.3431 

.3333 

.8812 

.9470 

.3740 

.3556 

.9114 

.9441 

- 24  - 


J 


T«ble  ^ - Continued 


Marginal  Ratio 

Pour-CateRorv  Desians 
Sample  Size  56 
eoual 

unequal 

Treatment  Effect 

,2 

.5 

.2 

Eta 



^ Ji a 

Li l2 

^ 

Hodel  Power  Coapar Isons 


RB/RC 


.01 

RB 

.0317 

.0290 

.2353. 

.2193 

.0320 

.0300 

.2173 

RC 

.0297 

.0303 

.2097 

.2107 

.0323 

.0280 

.2220 

.2;=4o 

Nonlnal 

.05 

RB 

.U13 

.IU3 

•‘*757, 

.U46o 

.1130 

.1183. 

.4547 

.4(^7 

Alpha 

RC 

.1070 

.1130 

.U380 

.U29O 

.1053 

.1020 

.4540 

.4;i03 

.10 

RB 

.1863 

.1983 

.6020. 

.56U7 

.1750 

.1917 

.5743 

.5900 

RC 

.1820 

.1893 

.?720 

.5503 

.1780 

.18:7 

.?703 

.3733 

RBOl/RCOl 

.01 

RB 

.0500 

.0667 

.3200 

.3636. 

.222^ 

.1290 

'.4231.. 

.3793 

RC 

.0000 

.0000 

.3000 

.1250 

.0000 

.0303 

.1143 

.2837 

Nonlnal 

.05 

RB 

.2000 

.1353 

.5200 

.9*55 

.1*074. 

.1290 

.3769 

.5862 

Alpha 

RC 

.0625 

.1000 

.5000 

.3000 

.0870 

.1212 

.4000 

.4286 

.10 

RB 

.2500 

.2000 

.6800 

•7727.. 

.5185, 

.1613 

.6154 

.7586, 

RC 

.0938 

.2000 

.6000 

.U250 

.2174 

.2424 

.7714 

.5000 

RB05/RC05 

.01 

RB 

.0268 

.0526 

.3116 

.?727„ 

.0671 

.06 

.2463 

.2500 

RC 

.0192 

.0129 

.2553 

• 135? 

.0355 

.0311 

.1667 

.2171 

Nominal 

.05 

RB 

.13>‘2 

.1382 

.5072 

.5035 

.2256 

.1168 

.5075 

.^05, 

Alpha 

RC 

.1026 

.0839 

.U61O 

.3871 

.15^ 

.0745 

.4058 

.3953 

,10 

RB 

.2081 

.2039 

.6522 

.6361*. 

.2988 

.1606 

.5696 

.6484. 

RC 

.1923 

.1871 

.6099 

.1903 

.2482 

.1739 

.5362 

.5116 

Saaple  Slee  224 


RB/RC 

.1337 

.1467 

.8743 

.8727 

.1237 

.1387 

.8730 

.8695 

.1363 

.1420 

.8740 

.8717 

.1230 

.1387 

.8633 

.8683 

Nonlnal 

■05  “ 

.3180 

.3340 

.9637 

.9660 

.3230 

.3173 

.0640 

.9633 

Alpha 

■ 3323 

.3267 

.9583 

.9630 

.3180 

.3313 

■ 9593 

.9640 

.4413 

.4617 

.9843 

.9840 

.4573 

.4417 

.9823 

.9637 

.4550 

.4497 

.9790 

.9810 

.4453 

.4453 

.9833 

.9813 

RBOl/ROOl 

.01  “ 
RC 

.1765 

.0690 

• 87/1 

• 9355 

.2083 

.1779 

.7917 

.7241 

.0417 

.1034 

.8333 

.8788 

.1538 

.1563 

•7727 

.8571 

Nominal 

.05 

^ RC 

.2941 

.2069 

1.0000 

1.0000 

.3355 

.24:4 

.9583 

.9310 

Alpha 

.2083 

• 1379 

1.0000 

■9697 

.^62 

.2500 

.9545 

.9286 

•10  ?r 

RC 

.3235 

.3448 

1.0000 

1,0000 

.4583 

.4138 

1.0000 

1.0000 

.3333 

.3448 

1,0000 

.9697 

.4615 

.3750 

•9545 

1.0000 

RB05/R005 

- 

.1400 

.679 

.6599 

.9167 

.1799 

.1419 

.8767 

•8553 

.0800 

.1678 

.9091 

.9006 

.1214 

•1594 

.8414 

.8130 

Nominal 

.2400 

■ 5212 

.9682 

.9848 

.3237 

.3378 

.9521 

.9623 

Alpha 

.2560 

.3221 

.9805 

• 9752 

.3500 

.3623 

.9517 

.9512 

-0  ll 

.3333 

.4380 

.9809 

• 9924 

.4676 

.4277 

.9795 

.9874 

.3680 

.4698 

.9935 

.9876 

.4286 

.4928 

.9862 

.9837 

* PeiMr  cemM'>*on  •tfntfleantlv  <Htf«r«nt  at  .OS  Wwai. 
' * Po<N*r  compartMn  ■tgnificantiv  diffaram  at  .01  laval. 


New:  ac  - aCANCOVA. 

asoi  ■ as  analvdt  In  Mdtkh  IntaracHon  N it^lftcant  at  the  .01  W«ai. 

asos  ■ as  analytH  in  adilcn  Intaraetion  N alfnlflcani  at  itw  .OS  tawi. 

acoi  • aCANCOVA  analyW*  In  whlah  aatarofwialtv  NtldnlfWant  at  the  .01  WmI. 

acOS  > ac ANCOV A anatyalt  In  wdtlca  hatarofanaltv  N atanlflcant  at  lha  .00  lawal. 

/ • Cemparad  iwith. 
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the  CANCOVA  technique  unlfonnly  more  powerful  under  these  conditions. 

This  relationship  does  not  hold^  however,  for  conditions  In  which 

SS  “ SS  is  not  equal  to  0.  A value  equal  to  the  ratio  SS,  / s 

h Y ’he 

PV,  was  defined  for  situations  in  which  Pw(RB)  = Pw(C).  PV  was  shown  to 
be  a monotonic  decreasing  function  of  sample  size.  The  smaller  sample 
sizes,  within  any  design  type,  allow  for  the  greatest  amounts  of  hetero- 
geneity (interaction)  before  Pw(RB)  > Pw(t);  in  addition,  the  degrees  of 
freedom  difference  between  RB  and  CANCOVA  techniques  [df(c)  > df(b)] 
provides  a higher  probability  for  rejection  of  the  null  hypothesis  of 
no  treatment  effect  under  CANCOVA.  These  combined  factors  provide 
CANCOVA  with  a power  advantage  when  the  sample  size  is  relatively  small 
and  heterogeneity  (interaction)  is  minimal.  As  the  sample  size  Increases, 
PV  approaches  0 and  at  the  same  time  the  degrees  of  freedom  advantage 
of  CANCOVA  over  RB  becomes  negligible.  Under  these  conditions  the  power 
advantage  tends  to  shift  in  favor  of  RB.  The  relatively  small  PV  of  the 
larger  sample  size  does  not  permit  very  high  levels  of  heterogeneity 
(interaction)  to  be  present  before  Pw(RB)  becomes  greater  than  Pw(C). 

Another  aspect  of  the  analytical  results  involves  the  implications 
regarding  the  relationship  between  power  and  the  correlation  between 
the  concomitant  variable  and  the  criterion  variable.  For  RB  and  ANCOVA 
designs  in  which  the  concomitant  variables  are  continuous,  the  relative 
power  of  each  design  is  dependent  on  the  value  of  the  linear  correlation 
coefficient  (p)  between  the  concomitant  and  criterion  variables.  When 
p < 0.4,  the  RB  technique  tends  to  be  more  powerful;  when  p > 0.6,  the 
ANCOVA  technique  tends  to  be  more  powerful.”  However,  these  results  do 
not  generalize  to  the  case  when  the  concomitant  variable  is  not  continuous. 
In  RB,  the  qualitative  concomitant  variable  is  used  as  a blocking  variable 
and  the  amount  of  error  reduction  is  equal  to  the  sum  of  squares  for 
blocking,  SS^;  In  CANCOVA,  the  qualitative  concomitant  variable  Is  used 

to  generate  a set  of  dichotomous  dutmny  covariables  and  the  amount  of  error 
reduction  Is  equal  to  the  sum  of  squares  for  the  pooled  within-cell 
regression,  SSj^.  It  was  shown  that  SS^  = SSj^;  since  the  amount  of  error 

variability  explained  by  the  correlation  between  the  qualitative  concomi- 
tant and  criterion  variables  for  both  the  RB  and  CANCOVA  analyses  Is 
the  same,  the  power  difference  between  the  RB  or  CANCOVA  techniques  is 
not  a function  of  the  value  of  eta. 


Empirical  Results 

Goodness-of-flt.  Using  chi-square  goodness-of-fit  tests.  It  was 
shown  that  all  12  null  design  combinations  produced  chi-square  statistics 
which  fell  within  confidence  intervals  for  chance  occurrences.  These 
sample  runs  were  used  to  establish  the  adequacy  of  the  random  number 
generators  and  to  empirically  validate  the  computational  subroutines 
within  the  Monte  Carlo  program. 


” Cox,  1957,  op.  cit.;  Feldt,  1958,  op.  clt. 
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Empirical  Power  Comparisons.  The  empirical  data  showed  no  Inter- 
pretable differences,  either  In  magnitude  and  direction,  between  the 
power  of  the  RB  and  RCANCOVA.  At  nominal  alpha  levels  of  .01,  .05,  and 
.10  the  number  of  significantly  different  power  comparisons  for  the 
two-,  three-,  and  four-category  design  types  fell  within  95^  confidence 
Intervals  for  chance  occurrences.  Empirically  there  Is  no  difference 
In  power  between  the  two  techniques;  however,  caution  must  be  exercised 
In  Interpreting  these  results  to  an  applied  situation.  RCANCOVA  was 
designed  as  a post-hoc  CANCOVA  In  which  the  a priori  fixed  number  of 
observations  per  category  of  the  concomitant  variable  (n^.)  becomes  a 

post  hoc  random  variable.  Since  CANCOVA  and  RB  were  shown  to  be 
analytically  equivalent,  RCANCOVA  could  be  viewed  as  a post  hoc  blocking 
technique;  however.  In  actual  practice  post  hoc  blocking  Is  a technique 
In  which  the  a priori  fixed  number  of  observations  per  cell  of  the 
design  (n^j)  becomes  a post  hoc  random  variable.  Since  n^^.  4-  the 

RCANCOVA  Is  not  equivalent  to  a post  hoc  blocking  technique;  further 
Investigation  using  more  realistic  post  hoc  models  Is  necessary  before 
Inferences  can  be  c.rawn  regarding  the  relative  power  of  RB  and  post 
hoc  RB  designs. 


SUMMARY 

The  powers  of  fixed-effects  randomized  block  (RB)  and  analysis  of 
covariance  (CANCOVA)  using  qualitative  concomitant  variables  were 
analytically  and  empirically  compared.  Analytical  comparisons  were  made 
of  the  powers  of  RB  and  CANCOVA  In  which  the  number  of  observations  (n  ) 
within  each  of  the  I categories  of  the  concomitant  variable  was  a 
constant.  Empirical  comparisons  were  made  of  the  power  of  CANCOVA  In 
which  n^  was  a random  variable  (RCANCOVA)  with  RB  in  which  n^^  was  a 

constant.  A Monte  Carlo  program  simulated  fixed-effects  analyses  with 
two  levels  of  treatment,  one  criterion  variable,  and  a qualitative 
concomitant  variable  with  I categories.  Three  design  types  In  which 
I was  equal  to  2,  3,  and  4 were  studied.  The  parameters  varied  for 
each  design  type  were:  (1)  total  sample  size  (n..)  (I“2,  n..«20,  80;, 
1=5,  n..*56,  1=4,  n..«56,  224),  (2)  ratio  of  number  of  row 

observations  (1=2,  1:1  4:1;  1=3,  1:1:1,  4:1:1;  1-4,  1:1:1:1,  4:1:1:1), 

(3)  eta  (0.0,  0.5,  0.9),  and  (4)  magnitude  of  treatment  effect  (O.O, 

0.2.  0.5). 

Analytically,  the  RB  and  CANCOVA  provided  the  same  Information  In 
terms  of  component  sums  of  squares.  However,  the  power  relationship  was 
shown  to  be  a function  of  sample  size,  design  type,  and  amount  of 
heterogeneity  (interaction)  present.  Empirically  no  interpretable 
differences  were  found,  either  in  magnitude  and  direction,  between  the 
power  of  the  RB  and  RCANCOVA  for  any  of  the  design  type  and  parameter 
combinations  studied. 
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APPENDIX  A 


DOCUMENTATION  FOR  PROGRAM  WHICH  COMPUTES  ETA 
FROM  MARGINAL  ROW  MEANS 


A single  parameter  card  Is  used  for  each  combination  of  marginal 
means  tested. 

Col  Information 

1 number  of  categories  In  the  control  variable  (2,  5,  or  4) 

2-4  value  of  marginal  mean  for  first  blocking  level  (F5.2) 

5-7  value  of  marginal  mean  for  second  blocking  level  (F5.2) 

8-l0  value  of  marginal  mean  for  third  blocking  level  (F5.2) 

11-15  value  of  marginal  mean  for  fourth  blocking  level  (F5>2) 

(Card  contains  only  as  many  means  as  there  are  categories,  l>e.,  a three  cate 
gory  control  variable  design  has  only  three  marginal  mean  values  punched.) 


The  program  reads  li,  the  marginal  mean  values  and  computes  the  value 
of  eta  based  on  the  following  relationship: 


kk 


where  I « number  of  rows,  n - row  mean,  pi  - grand  mean,  o£  Is  the 
total  variance  and  Is  set  equal  to  one. 

Output  consists  of  the  values  of  the  row  means  and  the  computed  eta. 

Table  A-1  shows  the  row  means  for  each  eta  used  In  the  Monte-Carlo 
analyses . 


Table  A-1 

ROW  MEAN  VALUES  FOR  EACH  DESIGN  TYPE 


Control  Variable  Designs 


Value 

Two  Category 

Three  Category 

Four  Category 

of 

Eta 

Row 

1 2 

Row 

1 2 

5 

Row 

125 

4 

.5 

1.00 

1.60 

1.00  1.50 

1.70 

1.00  1.20  1.40 

1.80 

■ 9 

1.00 

2.80 

1.00  1.60 

5.75 

1.00  1.60  2.20 

5.40 

55  - 


The  row  means  In  Table  A-1  were  used  to  compute  the  actual  cell 
means  for  each  cell  In  a specific  design.  The  computed  cell  means  were 
used  by  RANDN  to  generate  the  observations  within  that  cell.  For 
example,  If  the  value  of  eta  was  and  the  treatment  effect  was  .2  for 
a two  category  design,  then  cell  (ll)'s  population  mean  would  be  O.9O, 
cell  (12)'s  population  mean  would  be  l.lO,  cell  (21)'s  population  mean 
would  be  l.'iO,  and  cell  (22)'s  population  mean  would  be  I.70. 


APPENDIX  B 


DOCUMENTATION  FOR  THE  CHI-SQUARE 
GOODNESS-OF-FIT  TEST 


A single  parameter  card  was  used  for  each  starting  number  to  be 
tested. 

Col  Information 

1-6  six  digit  starting  number  read  In  under  F format  (F6.0) 

The  program  read  In  the  starting  number  and  called  up  the  subroutine 
RANDN  to  generate  1,000  numbers,  with  a mean  of  0.0  and  standard  deviation 
of  one.  The  program  divides  the  empirical  frequency  distribution  Into 
16  Intervals  of  .4  standard  deviations  each.  Observed  frequencies  are 
compared  with  Che  expected  for  each  Interval  and  chi  squares  for  each 
Interval  as  well  as  an  overall  chi  square  are  computed. 

Output  consists  of  Che  overall  chi  square,  observed  frequencies  for 
each  of  Che  16  Intervals  and  chi  squares  for  each  Interval. 
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APPENDIX  C 


DOCUMENTATION  FOR  MONTE  CARLO  PROGRAM 


The  following  parameter  cards  are  used  by  the  Monte  Carlo  program: 
(1)  S*'arting  Number  Card 

Col  Information 


1-6  six  digit  starting  number  for  RANDN  (F6.0) 

7-12  six  digit  starting  number  for  RAMDU  (f6.0) 

13-16  four  digit  field  for  number  of  sample  generations  to  be  run 

for  each  design  combination  read  In  on  the  following  parameter 
cards  (II^) 

(2) 

Col 

1 0 stop  program 

1 read  In  parameters  and  run 

2 one  digit  F field  for  the  value  of  eta,  (Fl.l) 

5-5  three  digit  F field  for  treatment  effect  difference,  (F3.2) 

6 one  digit  I field  for  number  of  cells  in  the  RB  analysis  k, 

6,  or  8 

7-8  two  digit  I field  defining  number  of  observations  to  be 

generated  by  RANDN  in  cell  (11) 

9-11  three  digit  F field  defining  the  value  of  the  population  mean 

under  which  the  observations  In  cell  (11)  will  be  generated 
by  RANDN,  (F3.2) 

12-13  Cwo  digit  I field  defining  number  of  observations  to  be 

generated  by  RANDN  In  cell  (21) 

11^-16  three  digit  F field  defining  the  value  of  the  population 

mean  under  which  the  observations  in  cell  (21)  will  be 
generated  by  RANDN,  (F3>2) 

Col 


repeat  until  parameters  for  all  cells  Involved  In  the  particular  design 
are  punched.  The  maximum  allowed  Is  for  a 4 x 2 design  with  8 cells. 


BUIOC-W* 
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The  program  reads  In  Che  parameter  cards;  card  (1)  Is  read  only  once 
for  each  set  of  runs.  There  is  no  limit  to  the  number  of  type  (2)  cards 
which  may  be  read  using  a single  set  of  starting  numbers.  If  It  Is 
desirable  to  use  different  sets  of  starting  numbers  for  each  design 
combination,  then  Che  program  must  be  terminated  after  each  run  and  a 
new  set  of  parameter  cards,  both  (1)  and  (2),  used  Co  start  a new  run. 

RANDN  is  called  up  to  generate  data  for  each  cell  of  Che  design. 

The  data  generated  are  analyzed  by  the  RB  subroutine.  A new  set  of  data 
Is  generated  by  RANDN  under  subroutine  RCANCOVA.  This  set  has  Che  same 
number  of  observations  as  the  RB  analysis,  but  the  number  of  observations 
per  control  variable  category  Is  determined  by  a subroutine  which  randomly 
assigns  category  classifications  to  each  observation  generated  by  RANDN. 
Ihe  probability  of  assigning  a set  of  covariable  scores  corresponding  to 
a given  category  of  Che  concomitant  variable  Is  the  ratio  of  the  number 
of  observations  In  that  category  to  the  total  number  of  observations 
for  Che  design.  RCANCOVA  produces  a set  of  data  In  which  the  number  of 
observations  In  each  category  of  the  control  variable  is  a random 
variable  and  Is  representative  of  Che  population  defined  in  Che  parameter 
card.  Each  observation  within  a category  Is  assigned  the  proper  set  of 
covari.able  scores  and  the  set  of  data  generated  under  RCANCOVA  Is  analyzed 
as  If  It  were  a one-way  ANCOVA. 

Output  consists  of  a table  containing  the  following; 

1.  The  design  parameters 

a.  The  dimensions  of  Che  design 

b.  Eta 

c.  Treatment  effect  difference 

d.  Cell  sizes,  treatment  level  sizes,  blocking  level  sizes,  and 
the  total  sample  size 

2.  Total  sample  daCa--suninary  data  for  all  6,000  samples  generated 

a.  Frequency  of  rejection,  proportion  of  rejection,  z and  z ' 
statistics  at  all  six  nominal  alpha  levels  for  the  RB  and 
RCANCOVA  analyses 

b.  A total  frequency  count  for  each  analytical  technique 

3.  Interaction  and  heterogeneity  at  the  .01  level — data  for  all 

samples  out  of  the  6,000  with  Interaction  In  the  RB  analysis 

significant  at  the  .01  level  and  heterogeneity  In  the  RCANCOVA 

analyses  significant  at  the  .01  level 

a.  Same  as  2a  above 

b.  Same  as  2b  above 
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4.  Same  as  3 above  except  for  Interaction  and  heterogeneity  which 
was  significant  at  the  .0^  level. 
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